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This is in response to the appeal brief filed 6/2/2005. 
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(1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly 
affect or be directly affected by or have a bearing on the decision in the pending appeal 
is contained in the brief. 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is incorrect. A 
correct statement of the status of the claims is as follows: 

Claims 6 and 15 are now objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

This appeal now involves claims 1-5, 7-14, and 16-36. 

(4) Status of Amendments After Final 

No amendment after final has been filed. 

(5) Summary of Invention 

The summary of invention contained in the brief is correct. 

(6) Issues 

The appellant's statement of the issues in the brief is correct. 

(7) Grouping of Claims 
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The rejection of claims 1-36 stand or fall together because appellant's brief does 
not include a statement that this grouping of claims does not stand or fall together and 
reasons in support thereof. See 37 CFR 1.192(c)(7). 

(8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(9) Prior Art of Record 

5,774,595 Kim 6-1998 

5,974,175 Suzuki 10-1999 

4,843,630 Catrosetal. 6-1989 

6,332,034 Makram-Ebeid et al. 12-2001 

6,055,337 Kim 4-2000 

5,471,535 Ikezawa 11-1995 

Huitao Luo and Alexandra Eleftheriadis, "Desiging an Interactive Tool for Video 
Object Segmentation and Annotation," Advent Group, Columbia University, July 12, 
1999. 

(10) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
Claims 1-5, 7-14, and 16-36 are rejected under 35 U.S.C. 103(a). This rejection 
is set forth in a prior Office Action, mailed on 12/27/2004. 

(11) Response to Argument 
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A. As to the argument that the rejection of claims 1-19, 25-30, and 33-36 
over Kim ('595) and Suzuki is improper because there is no motivation to 
combine the references: 

Appellant argues that Kim and Suzuki are directed to different solutions for 
different problems. Appellant further argues that the Kim reference is a video signal 
encoder capable of reducing approximation errors in encoded contour information, while 
Suzuki is directed towards identifying a contour for extraction of an object from an 
image frame. Appellant further argues that because they are directed to different 
subjects, the Suzuki reference cannot be used to improve Kim, and in fact would 
destroy the Kim reference because user input would slow the video encoding of Kim 
unacceptably. 

In response, the examiner notes that both Kim and Suzuki are directed towards 
correctly fitting contours in a moving picture sequence. In col. 2, lines 7-10, Kim 
discloses "...determining a number of vertices on the contour image... [and] fitting the 
contour image with a plurality of line segments to provide a polygonal approximation of 
the contour image..." Similarly, in col. 2, lines 57-59, Suzuki discloses "...a first contour 
detection unit which detects contour points which determine a contour of the object of 
the first image..." Contour detection may be used for different purposes in each 
invention, but both inventions do include this feature and therefore the combination of 
Kim and Suzuki is not one of inventions from drastically different arts. It is further noted 
that both the Kim and Suzuki references are found primarily classified at USPTO in 
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class 382, with both cross-referenced to subclass 266. Thus, the Office has found at 
least some of the teachings of Kim and Suzuki to be similar. 

As to the argument that the teachings of the Suzuki reference would slow the 
invention of Kim unacceptably, the examiner points out that Suzuki does not require 
user input in every frame of video. In fact, Suzuki requires only one user input for an 
object in a frame, and then automatically detects contours in subsequent frames. Col. 3, 
lines 34-37 of Suzuki state "Only the frame and object boundary specification for the 
first frame is given by the operator, and the contour detection for all the frames is 
automatically carried out by the computer operations". Certainly, a user input on a 
single frame would not slow encoding of frames by an amount that would be 
unacceptable. 

Even assuming the appellant is correct that the Suzuki teachings would slow the 
invention of Kim, prior case law has stated that inefficiency does not preclude a 
reference or combination of references from being used against an applicant. See 
Orthopedic Equip. Co. v. United States, 702 F.2d 1005. 1013. 217 USPQ 193, 199-200 
(Fed. Or. 1983). While having a user select contours during each frame of video 
encoding would not be a particularly efficient way to encode a large amount of video, it 
would be technically possible. 

The examiner further notes that prior case law has stated that making a process 
automatic vs. making a process manual is not a patentable distinction. See In re 
Venner, 120 USPQ 192 (CCPA 1958). Even assuming that the appellant is correct in 
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arguing that Kim and Suzuki cannot be combined, Kim alone would be sufficient to 
anticipate appellant's invention, because Suzuki only adds a manual interface. 

The motivation for a combination of Kim and Suzuki is laid bare in col. 1 , lines 
10-14 of the Suzuki reference: "extracting a desired object from a sequence of images 
of a motion picture on the basis of a detected contour of the object while requiring only a 
simple operation from the operator". Without user input, the encoder of Kim suffers 
from the deficiency of not being able to select a single object for extraction. The 
invention of Suzuki corrects this deficiency. 

B. As to the argument that the rejection of claims 1-19, 25-30, and 33-36 
over Kim ( ( 595) and Suzuki is improper because there is no reasonable 
expectation of success: 

As in arguments covered under heading A, appellant argues that significant user 
intervention into the encoding process would be required if the teachings of Suzuki were 
applied to Kim. Appellant further argues that this intervention would destroy the 
purpose of Kim by slowing processing speeds, citing case law (In re Fitch) that states 
modification is inappropriate when it renders prior art inoperable for its intended 
purpose. Examiner notes that Suzuki requires only one user input for an object in a 
frame, and then automatically detects contours in subsequent frames. Col. 3, lines 34- 
37 of Suzuki state "Only the frame and object boundary specification for the first frame 
is given by the operator, and the contour detection for all the frames is automatically 
carried out by the computer operations". Certainly, a user input on a single frame would 
not slow encoding of frames by an amount that would be unacceptable. 
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Even assuming appellant's argument that the modification to Kim suggested by 
Suzuki would slow the Kim invention considerably is correct, the Kim invention would 
still achieve its purpose of video encoding. Prior case law has stated that inefficiency 
does not preclude a reference or combination of references from being used against an 
applicant. See Orthopedic Equip. Co. v. United States, 702 F.2d 1005. 1013. 217 
USPQ 193, 199-200 (Fed. Cir. 1983). 

C. As to the argument that the rejection of claims 12-19, 29-30, and 35-36 
over Kim ('595) and Suzuki is improper because positively recited limitations of 
the claims are not disclosed by the prior art even if the references are combined: 

Appellant argues that neither Kim nor Suzuki teaches encoding into a "data 
structure" which is converted. It is the Office's position that "data structure" is a broad 
term that is not further defined by the claim. A broadly defined "data structure" can 
encompass any method of storing data, such as in the coded set of quantized transform 
coefficients in col. 2, lines 19-25 of Kim. These coefficients are data to be transmitted, 
and therefore have some sort of associated structure inherent. Transmitted data 
without a structure would be unacceptable because without a structure, data is not 
organized and cannot be read. Even using a stricter definition, such as that for the 
computing term "data structure" at the NIST website 

(http://www.nist.gov/dads/HTMLydatastructur.html), which gives the definition of a "data 
structure" as "An organization of information, usually in memory, for better algorithm 
efficiency, such as queue, stack, linked list, heap, dictionary, and tree, or conceptual 
unity, such as the name and address of a person. It may include redundant information, 



Application/Control Number: 10/046,797 Page 8 

Art Unit: 2676 

such as length of the list or number of nodes in a subtree", Kim still meets the definition 
of a "data structure". An "organization of information" is inherent to any data transmitted 
with intent to be decoded. 

Appellant further argues that "converting graphical information of the data 
structure from a first format to a second format different from the first format" is not well- 
known in the art. The examiner notes that any number of common graphics programs 
such as Microsoft Paint or Adobe Photoshop can perform a format conversion from 
JPEG to bitmap, or from JPEG to GIF. Similarly, many well-known video editors can 
convert MPEG2 to MPEG or vice versa. It is nearly impossible to buy a computer that 
does not have some sort of graphical conversion software on it, as Microsoft Paint 
comes standard with Microsoft Windows XP. Graphical conversion is extremely well- 
known in the art, so well-known that even those not skilled in the art are well-versed in 
its use. 

D. As to the argument that the rejection of claims 33-34 over Kim ('595) and 
Suzuki is improper because positively recited limitations of the claims are not 
disclosed by the prior art even if the references are combined: 

Appellant argues that claim 33 recites a user interface configured to receive user 
input and processing circuitry and configured to detect contours between respective 
pairs of vertices responsive to respective user input received via the user interface and 
that this limitation is not addressed in the grouped rejection of claims 1 and 33. The 
examiner notes that claims 1 and 33 were rejected with disclosure from Suzuki that 
determines a contour "when the first image and points adjacent boundary locations of 
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the object of the first image are specified by a user" (col. 2, lines 59-61). A user 
interface is inherent to Suzuki, because Suzuki's invention is on a computer and an 
interface is the only way for a user to input to a computer. Hence, the "user interface" 
portion of the limitation of claim 33 can be grouped with the "user input" portion of claim 
1 . "Processing circuitry to detect contours between respective pairs of vertices 
responsive to respective user input received via the user interface" is a more specific 
apparatus than in claim 1, but because "processing circuitry" is inherent to any 
computerized device, and said circuitry is performing the same steps as in claim 1, this 
limitation can also be grouped with the second limitation of claim 1 . 

E. As to the argument that the rejection of claims 25, 29, and 34 over Kim 
('595) and Suzuki is improper because positively recited limitations of the claims 
are not disclosed by the prior art even if the references are combined: 

. Appellant argues that Suzuki does not teach a system where user input is 
different for each contour, because Suzuki uses automatic detection after one contour 
has been input by a user. The examiner agrees that automatic detection is used but 
believes that the rejection of claim 25 has been misinterpreted by appellant. In fact, 
Suzuki teaches a system where one contour is detected based on user input and that 
single contour is then automatically detected in subsequent frames (col. 2, lines 53-65). 
If one wanted to select a different contour, one would have to restart the invention of 
Suzuki and select a second contour manually. This restarting of the invention is not 
disclosed in Suzuki. However, it is inherent to Suzuki that if the invention were 
restarted, a second contour with different user input than the first would be selected. 
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F. As to the argument that the rejection of claim 28 over Kim ('595), Suzuki, 
and Ikezawa is improper because there is no motivation to combine the reference 
teachings: 

Appellant argues that there is no motivation to look for references to add to Kim 
and Suzuki because there is no evidence that Kim or Suzuki suffers from an inability to 
process "complicated shapes". The examiner notes that Ikezawa provides an 
improvement over the Kim and Suzuki references and includes proper motivation that 
renders an obvious combination, in this case the ability to better process complicated 
shapes by allowing user editing, as stated by col. 1 1 , lines 61-63. 

G. As to the argument that the rejection of claim 4 over Kim ('595), Suzuki, 
and Catros is improper because there is no motivation to combine the reference 
teachings: 

Appellant argues that one concerned with video encoding would not look to 
teachings about bridging contours and therefore the combination of Catros with Kim and 
Suzuki lacks proper motivation. The examiner notes that all three references are cross- 
referenced to class 382, subclass 266 in the USPTO database, so they have been 
previously judged to include similar material. Further, all of Kim, Catros, and Suzuki are 
concerned with fitting contours. Figures 2a-2c of Kim show a process for approximating 
a contour of an object with straight lines. This process is further explained in col. 3, 
lines 10-30 of Kim. Suzuki also approximates a contour, as explained in col. 5, lines 53- 
64: "The contour detection unit... operates to detect a contour of the desired object in 
the image of the specified frame..." Finally, Catros searches for a best-fit contour 
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between two points that do not yet have a contour defined between them. Col. 2, lines 
21-29 of Catros state "...the contours thus obtained and stored in digital image memory 
have discontinuities and it may be desirable to fill these in..." 

The Kim reference does involve encoding contours as described in appellant's 
brief. However, if one were to use Kim's invention on an image missing contours, one 
would clearly look for a solution to fill in these missing contours. Catros provides this as 
motivation for using the invention. As stated in the above paragraph, it may be 
desirable to fill in discontinuities in an image if such discontinuities exist. Because both 
references are cross-referenced to the same subclass and both are concerned with 
finding contours, they are sufficiently similar enough to be combined. Because the 
Catros reference provides an advantageous feature not included in the Kim reference 
and clearly states the desirability of this feature in a contour-detection system, sufficient 
motivation has been provided for a combination between the two references. 

H. As to the argument that the rejection of claim 35 over Kim ( c 595) and 
Suzuki is improper because there is no motivation to combine the reference 
teachings: 

Appellant argues that Kim is concerned with compression of data and not 
concerned with extraction of a graphical image, and therefore a combination between 
Kim and Suzuki is improper and lacks motivation. The examiner notes that, in addition 
to compression of data, Kim is concerned with fitting and encoding contours, as 
evidenced by figures 2a-2c and col. 3, lines 10-30 of the reference. As stated in 
previous Office Actions, Kim does not disclose extraction of contour information. 
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Suzuki, however, discloses that extraction of contour information is advantageous for 
easy manipulation and editing of images in col. 1, lines 10-24. This text including the 
extraction feature and proper motivation for combination with Kim was cited in the 
rejection to claim 12 on page 7 of the non-final Office Action mailed December 27, 
2004. As claim 35 is dependent on claim 12, and proper motivation was already cited in 
the rejection to claim 12, no further motivation rationale is necessary for the rejection to 
claim 35. 

I. As to the argument that the rejection of claims 20-24 and 31-32 over 
Catros and Makram-Ebeid is improper because there is no motivation to combine 
the references: 

Appellant argues that the solutions provided by Makram-Ebeid are not concerned 
with the problems of Catros. Appellant further argues that the examiner's argument that 
regions and contours are closely related is not supported by evidence. Appellant 
concludes that Catros and Makram-Ebeid are not compatible inventions and insufficient 
motivation exists to add the region merging teachings of Makram-Ebeid to the contour- 
fitting teachings of Catros. The examiner notes that the concept of contours defining 
regions is well-known in the art. The appellant's claim 20 includes a recitation of 
"regions defined by... respective contours". Answers.com recites the definition of a 
contour as "the outline of a figure, body, or mass". Region is defined as "a large, 
usually continuous segment of a surface or space; area". It is elementary that the 
outline of a figure would define a continuous segment of space. The concepts of 
regions and contours are so dependent on each other that it would be impossible to 
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define a region without some sort of contour that surrounds it. Otherwise, the region 
would be indefinite. 

Continuing to the references themselves, the examiner notes that figures 5a and 
5b of Makram-Ebeid feature an image segmented into regions (or cells, as referred to 
by the disclosure). These regions are surrounded by outlines, which are, as described 
above, synonymous with contours. Further, Makram-Ebeid discloses a function that 
takes into account total length of boundaries, another synonym for contours, in col. 1 , 
lines 48-53. Clearly, Makram-Ebeid is concerned with contours. In fact, they are a 
major portion of the reference's disclosed Energy function. The Catros reference 
discloses zones, a synonym for regions, in col. 1, lines 1 1-29. The image disclosed by 
Catros is made up of zones and contour lines. It is further noted that class 382, 
subclass 130 was searched in the USPTO database for both references, thus signifying 
that the Office considered references similar to the Catros reference when deciding on 
Makram-Ebeid's patentability. 

Makram-Ebeid discloses motivation for using a region merging method with a 
scale parameter in col. 1, lines 37-45, stating that it "eliminates the largest possible 
number of interfaces to merge adjacent regions whose intensities are practically 
identical". Applying the teachings of Makram-Ebeid to Catros would eliminate a number 
of contours (interfaces, boundaries, etc.), making Catros's goal of bridging contours 
easier, as the number of contours to bridge would be reduced. 
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J. As to the argument that the rejection of claims 20-24 and 31-32 over 
Catros and Makram-Ebeid is improper because positively recited limitations of 
the claims are not disclosed by the prior art even if the references are combined: 

Appellant argues that neither Makram-Ebeid nor Catros discloses weighting the 
respective shortest path by gradient calculations or association of contours with a 
respective scale parameter. Appellant further argues that neither reference discloses 
determining a scale parameter that minimizes variances between regions defined by its 
respective contours. On the contrary, Catros does disclose a method of weighting a 
respective shortest path by gradient calculations. As explained in pages 9-10 of the 
non-final Office Action mailed December 27, 2004, Catros discloses using gradient 
"amplitudes" in col. 2, lines 42-58. In image processing, a gradient measures change of 
color in an image on a given scale. Gradients are often used for contour detection 
because areas of an image where the largest amount of change takes place are likely to 
be contours that divide regions. The amplitude of a gradient is a representation of how 
much an image has changed from one area to another. In col. 2, lines 42-58 of Catros, 
the gradient amplitudes are used in an application of the Moore-Dijkstra algorithm to 
determine the best path to connect a disjointed contour. Because these amplitudes are 
numerical values associated with a gradient that are used to find a shortest path, they 
read on weight values. "Weight values" is considered a broad term because it is not 
further defined in claim 20, and thus, any numerical value that "weights" a path one way 
or another can read on a weight value. 
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With respect to contours associated with scale parameters, col. 1 , lines 53-59 of 
Makram-Ebeid discloses that interfaces (i.e. boundaries or contours, see col. 1, line 30 
of Makram-Ebeid) are eliminated when a scale parameter increases. Thus, each 
eliminated interface has an associated scale parameter or a plurality of associated scale 
parameters at which it exists. With respect to minimizing variance between regions, col. 
1, lines 48-53 of Makram-Ebeid discloses an Energy function that "takes into account 
the intensity variance in each region". When two regions are merged, this signifies that 
the Energy function is minimized, and since intensity variance is a key portion of 
Energy, it is also most likely at a minimum. Since claim 20 broadly recites "variance", it 
is assumed that "intensity variance" reads on such language. 

K. As to the argument that the rejection of claim 24 over Catros, Makram- 
Ebeid, and Luo is improper because there is no motivation to combine the 
reference teachings: 

Appellant argues that the modification of Catros to include a width parameter 
selected by user interface is contrary to the teachings of Catros. The examiner notes 
that col. 2, lines 63-68 and col. 3, lines 1-2 of Catros teach a width parameter of length 
D, as described in the rejection to claim 23 on pages 19-20 of the non-final Office Action 
mailed December 27, 2004. The examiner further notes that this disclosure describes a 
search space defined automatically using the coordinates of points A and B. 

In p. 8, lines 1-8, Luo also discloses a width parameter, in the form of a "search 
stripe", which allows a user to limit the height and width of an area where contours can 
be located. In p. 1, lines 27-29, Luo discloses that fully automatic segmentation of 
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images is difficult, giving the motivation to add a user interface to the invention of 
Catros. To make a task less difficult is a common motivation for combinations under 35 
USC 103(a), and it is explicitly disclosed that the user interface in the Luo invention 
achieves this. 

The examiner further notes that prior case law has stated making a process 
automatic vs. making a process manual is not a patentable distinction. See In re 
Venner, 120 USPQ 192 (CCPA 1958). Even assuming that the appellant is correct in 
arguing that Catros and Luo cannot be combined, Catros alone would be sufficient to 
anticipate appellant's invention, because Luo only adds a manual interface. 

For the above reasons, it is believed that the rejections should be sustained. 



Respectfully submitted, 



Aaron Richer 
August 18, 2005 
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